Method and system for inspecting unidentified mixed parts at an inspection station having a measurement axis to identify the parts

ABSTRACT

A method and system for inspecting unidentified mixed parts to identify the parts are provided. The system includes an inspection station which includes a vision-based robotic subsystem including a robot configured to pick up a self-supporting part and place the part on a fixtureless rotary stage so that a part axis of the part is substantially centered and aligned with a measurement axis of the inspection station during part rotation. A second subsystem optically measures a profile and features of the self-supporting part during part rotation. A processor is operable to compare the profile and the features of the part to be identified with the profile and corresponding features of stored templates to identify a template which matches the profile and features of the part to be identified and to generate and transmit an identification signal representing a part identification code for the part associated with a matched template.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. patent applications entitled“Computer-Implemented Method of Automatically Generating InspectionTemplates of a Plurality of Known Good Fasteners” and “High-Speed Methodand System for Inspecting and Sorting a Stream of Unidentified MixedParts” both of which are filed on the same day as this application.

TECHNICAL FIELD

At least one embodiment of the present invention generally relates tonon-contact methods and systems for identifying mixed parts and, inparticular, to optical methods and systems for identifying unidentifiedmixed parts at an inspection station.

OVERVIEW

Aircraft Construction

In the world of aircraft construction and repair, aircraft fasteners areutilized to assembly detail parts that are combined together with otherparts into assemblies, which are assembled into installations thatfinally end up as a complete aircraft. Typical threaded and non-threadedfasteners used in the aerospace industry are illustrated in FIG. 1.

As used herein, the term “fastener” refers to a hardware device thatmechanically joins or affixes two or more structures together. Forexample, a fastener may join two or more structures together. A fastenerincludes, for example, without limitation, a bolt a nut, a stud, ascrew, a rivet, a washer, a lock washer, and other suitable elements.

Fastener information may be found in various sources which may includethe specific aircraft maintenance manual, the specific aircraftstructural repair manual, or aircraft production and repair drawings.

Fastener Utilization and Types

When engineers design an aircraft many things are considered whenchoosing the correct type of fastener. The type of joint the fastenerwill be exposed to in its application; the shear or tension. What typesof loads will be transferred through the joint. Aircraft loads mayinclude those experienced during towing, normal flight operations,wind-gusts, pressurization, engine-out operations, landing, and more.All of this will determine how thick or thin the structure will have tobe, the material type of the original structure, and the associatedfasteners.

Fasteners must be able to achieve the transfer of load from one part toanother. An example of this is the load transferred from an engine to apylon, the pylon load to the wing, and the wing to fuselage. Fastenernumbers and diameter are calculated to transfer this load. Othercriteria are also needed to select the best fastener for theinstallation. This could include weight, inspect ability, toolingrequirements, aerodynamic smoothness, access, corrosion protection, andof course cost.

Fasteners can be placed into many groupings which may be used asstructural fasteners that take aircraft loads, to nonstructuralfasteners that connect non-load bearing parts. There are restrictedaccess applications or blind fasteners such a Huck Lock Bolts,Composi-Locks Fasteners, and CherryMAX fasteners.

For areas with access to both sides, standard rivets, structural bolts,and Hi-Lok fastening system fasteners are used. Materials for thesefasteners include aluminum, steel, and titanium and are coated toprevent dissimilar metal corrosion. Styles of fastener heads vary fromcountersunk to protruding head based on the aerodynamic requirements ofthe aircraft.

Fastener Codes and Orientation

Fastener coding can be designated by the fastener manufacturer such as aCR 3233 CherryMAX rivet, by an industry standard such as AN4 bolt, or bythe airframe manufacturer as in a BACR15CE5D3 rivet from Boeing. Codingdescriptions can be found in various fastener books, on the repairdrawing, or on the production blueprint. Also, fastener codes may beused to simplify repair and productions drawings.

It is standard for the head of a fastener to be installed head-up, orhead-forward. However, the blueprints and repair drawing will giveproper orientation in the fastener quadrant, and will call-out thefastener head near or far based on the view of the drawing.

One problem associated with fasteners such as aerospace fasteners isthat they become decertified and/or are mixed up with other fasteners.Many of those fasteners can be used after being recertified and,subsequently, re-introduced into inventory, thereby eliminating waste.

Traditional manual, gauging devices and techniques have been replaced tosome extent by automatic inspection methods and systems. For example,gage wires are utilized in physical thread measurements of pitchdiameter in the prior art. Two wires are placed in adjacent threads onone side of the UUT, and a single wire is placed on the other side ofthe UUT. A micrometer measures the distance between the reference lineestablished by the two adjacent gage wires and the reference pointestablished by the other gage wire. A tabulated correction formulaconverts the micrometer distance to an estimate of the pitch diameter.As in the readily appreciated, such inspection can be tedious, timeconsuming and prone to human error.

U.S. Pat. No. 7,633,635 discloses a method and system for automaticallyidentifying non-labeled, manufactured parts. The system includes anelectronic storage device to store templates of a plurality of knowngood, manufactured parts. Each of the templates includes a part profileand a set of features. Each of the features includes a range ofacceptable values. Each of the templates has a part identification codeassociated therewith. A first subsystem optically measures a profile andfeatures of a part to be purchased. The system further includes aprocessor operable to compare the profile and the features of the partto be purchased with the profile and corresponding features of each ofthe stored templates to identify a template which most closely matchesthe profile and features of the part to be purchased and to generate andtransmit an identification signal representing the part identificationcode for the part associated with the most closely matched template.

Other U.S. patents related to at least one embodiment of the presentinvention include: U.S. Pat. Nos. 7,403,872; 7,777,900; 8,237,935;7,633,046; 7,633,634; 7,738,121; 8,132,802; 8,550,444; 7,755,754;8,013,990; 7,738,088; 7,907,267; 7,796,278; 7,684,054; 7,920,278;7,812,970; 8,004,694; 8,416,403; 8,570,504; 9,047,657; 9,019,489;9,575,013; 9,697,596; 8,390,826; 8,896,844; 9,486,840; 8,993,914;9,372,077; 9,377,297; 9,228,957; 9,539,619; 9,370,799; 9,372,160;8,977,035; 10,209,200; and 10,207,297.

U.S. Pat. No. 10,207,297 discloses a method and system for inspecting amanufactured part supported on an optically-transparent window of arotary actuator at an inspection station. The window rotatably supportsthe part in a generally vertical orientation at which a bottom endsurface of the part has a position and orientation for opticalinspection. An illuminator is configured to illuminate the bottom endsurface of the part through the window with radiant energy to obtainreflected radiation signals which are reflected off the bottom endsurface of the part. The reflected radiation signals travel through thewindow. A lens and detector assembly is configured to form a bottomimage from the reflected radiation signals at a bottom imaging locationbelow the window and is configured to detect the bottom image. Thewindow is made of a material which is substantially transparent to theradiant energy and the reflected radiation signals.

Several types of algorithms or methods for detection of part defectshave been developed over the years. These algorithms and the systemsutilizing same can be classified into three general categories:Reference system pattern matching; non-reference or generic propertyverification; and measurement or gauging.

Reference systems compare pixel values or properties of a known goodparts with those under test. The matching may be done by directcomparison of intensity values, by comparison of statistical or spatialfeatures, or by matching nodes and end points in a graph which definesthe topology of the part. Mismatches between the reference patterns andthe image under test are used to detect flaws. One advantage of thereference approach is that much of the knowledge required to determineif a part is good is contained within the reference pattern. One of theproblems with this approach is the difficulty in accurately registeringor aligning the two images for comparison, thereby resulting in anambiguity range between a good and a defective part. Another problemwith the reference approach is that the data representation of thereference image must be sufficiently compact to avoid excessive memorycosts and slow data transfer rates. In general, the reference method isgood in finding gross defects but has more trouble in detecting flawssmall in size relative to the entire image because of themisregistration problem.

Non-reference systems differs from reference systems in that nocomparison between the image to be inspected and a reference image isneeded. This approach does not average the various features in an imageinto a single statistic or set of statistics but instead analyzes eachof the features individually. Thus, an important advantage of thisapproach is that no information is lost. Such a system searches an imagefor the presence of a specific set of features. Flaws are detected bythe presence or absence of these features. In order for such a system tobe successfully implemented, the set of guidelines used to detect ordescribe any given feature must be unambiguously defined and must beapplicable throughout the entire image being inspected.

The non-reference method is most effective in applications where a setof rules applies to each point in the part. If this method is to beexclusively used, then the inspection criteria must be described by acompact, context-independent set of specifications. Unfortunately,design rules are routinely violated throughout the part manufacturingindustry without affecting the functionality of a part design. Withrespect to certain designs, the presence of multiple layers can causethe appearance of violations when in fact no error has occurred. Theimpact on the non-reference method is that the list of inspection rulescan potentially become very long, resulting in a time-consumingalgorithm. In summation, the non-reference method is not general enoughto be applicable under a wide variety of conditions and often becomes“crippled” when rule violations occur.

Measurement or gauging systems are often designed to report dimensionsof patterns and provide feedback (S, Y and theta measurements foralignment) for process control. These systems assume that themeasurement area is defect free and that a good estimate, at least in aleast squares sense, is available to locate the position of each patternprecisely. Most commercially available part inspection systems do notprovide full-function part inspection and dimensional measurement. Forhybrid inspection systems alignment is very important and in integratedpart inspection systems precise line width measurements are required.Measurement accuracy at any single location is also affected bydigitization. Confidence in the measurement is typically plus or minusthe size of each pixel or picture element.

Many of the above-noted methods and systems use templates of known, goodparts during the inspection process. Templates for threaded fastenerstypically include a thread model. A thread model is an estimate for onecycle of the repeated thread form and is typically learned at templateedit time.

As described in U.S. Pat. No. 7,796,278 (i.e. '278 patent), A threadmodel is a learned sequence of points that represent a best estimate ofthe outline of one cycle of the thread form. The thread model iscalculated when the inspection region is specified, at template edittime.

The measure template routine uses a pattern match algorithm with a sinewave pattern to identify periodicity in the inspection region data. Thisprocess determines an approximate thread pitch. The process alsocalculates a starting point in the data vector for the first beginningof the matched pattern, which is an approximation to the first midpointof a right flank line.

With the pitch and the starting point in hand, the measure templateroutine can then calculate an average thread model. Starting with thefirst sample point in the matched pattern, points that are 1, 2, 3, . .. , N pitches later in the inspection region are averaged to form thefirst point of the thread model. The process is repeated for all therest of the points in the first matched pattern. The thread model isthen stored in the template for later use.

The thread model described in the '278 patent is a sampledrepresentation of one sensor's thread profile, for exactly one pitch.The thread model starts at the midpoint of a rising thread flank andends one pitch later.

Using a correlation detector the thread model is matched to data withinthe inspection regions, producing thresh-holded detections within theinspection region, that are called crossings. FIG. 17 of the '278 patentshows a sketch of a thread model matched to the sensor data.

The '278 patent also discloses a 3-Point Wire Pitch Diameter method. The3-point pitch diameter computes 3-point distances using the wirepositions computed in the sensor data. The 3-point wire pitch diameteris the median of the 3-point wire pitch diameter. FIG. 20 of the '278patent is a schematic view that illustrates a 3-point distance method,applied to thread wire positions. Shown are two wire positions in thetop thread form with a reference line drawn between them. Also, shown isa single wire position on the bottom thread form with the 3-pointdistance indicated.

As described in the '278 patent, the building or construction oftemplates for optical part inspection is tedious, time consuming andprone to human error. Consequently, it would be desirable if suchtemplates could be generated automatically without little or no userinput.

SUMMARY OF EXAMPLE EMBODIMENTS

An object of at least one embodiment of the present invention is toprovide a high-speed method and system for inspecting unidentified mixedparts to identify the parts rapidly and reliably. In this way, themethod and system can help recertify previously uncertified parts suchas fasteners so the recertified fasteners can be reintroduced intoinventory thereby reducing waste.

In carrying out the above object and other objects of at least oneembodiment of the present invention, a method of inspecting unidentifiedmixed parts at an inspection station having a measurement axis isprovided. The method includes storing templates of a plurality of knowngood parts. Each of the templates include a part profile and a set offeatures. Each of the features includes a range of acceptable values.Each of the templates has a part identification code associatedtherewith. At the inspection station, the method further includesproviding a vision-based robotic system including a robot to pick-up aself-supporting part to be identified and place the part on afixtureless rotary stage so that a part axis of the part issubstantially centered and aligned with the measurement axis of theinspection station. The method further includes optically measuring aprofile and features of the rotating part and comparing the profile andthe features of the part to be identified with the profile andcorresponding features of the stored templates to identify a part whichmatches the profile and features of the part to be identified. Stillfurther, the method includes generating and transmitting anidentification signal representing the part identification code for thepart associated with a matched template.

Unidentified parts and parts having an unacceptable geometric dimensionor defect may be directed to a reject part area and identified partshaving acceptable geometric dimensions and no significant defects may bedirected to good part areas based on the identification signals. In thisway, the parts are sorted.

The step of optically measuring may include the steps of projecting abeam of radiation at the self-supporting part and rotating the rotarystage and the part to be identified about the measurement axis so thatthe rotating part stays upright to partially obstruct the beam to obtainat least one unobstructed portion of the beam of radiation. The methodmay also include imaging the at least one unobstructed portion of thebeam of radiation to obtain a first set of electrical signals.

The method may further include processing the first set of electricalsignals to obtain the profile and features of the part to be identified.

The part to be identified may be at least partially conductive orsemiconductive, wherein a feature of at least one of the templatesincludes an eddy current signature. The method may further include thesteps of inducing an eddy current in the rotating part, sensing theinduced eddy current and comparing the eddy current signature with thesensed eddy current.

The part to be identified may partially obstruct the beam of radiationto obtain first and second unobstructed portions. The first and secondunobstructed portions of the beam of radiation may be imaged to obtainthe first set of electrical signals.

Each of the first and second unobstructed portions of the beam ofradiation may contain a magnitude of radiation which is representativeof a respective geometric dimension of the part to be identified.

The part to be identified may have threads. A last one of the templatesmay include at least one feature related to threads.

The part may be an externally threaded fastener.

Each part axis may be defined as being central to its part and parallelto its length.

Further in carrying out the above object and other objects of at leastone embodiment of the present invention, a system for inspectingunidentified mixed parts at an inspection station having a measurementaxis is provided. The system includes an electronic storage device tostore templates of a plurality of known good parts. Each of thetemplates includes a part profile and a set of features. Each of thefeatures includes a range of acceptable values. Each of the templateshas a part identification code associated therewith. At the inspectionstation, there is included a vision-based robotic subsystem including arobot configured to pick up and place a self-supporting part on afixtureless rotary stage so that a part axis of the part issubstantially centered and aligned with the measurement axis. A secondsubsystem optically measures a profile and features of the part duringpart rotation. A processor is operable to compare the profile and thefeatures of the part to be identified with the profile and correspondingfeatures of the stored templates to identify a template which matchesthe profile and features of the part to be identified and to generateand transmit an identification signal representing the partidentification code for the part associated with the matched template.

A mechanism including a part sorter may direct unidentified parts andparts having an unacceptable geometric dimension or defect to a rejectpart area and may direct identified parts having acceptable geometricaldimensions and no significant defects to good part areas based on theidentification signals.

A system controller may be coupled to the processor and the part sorterto control the sorting based on the inspecting.

The second subsystem may include a projector to project a beam ofradiation and a rotary actuator assembly to rotate the stage and theself-supporting part about the measurement axis so that the rotatingpart stays upright to partially obstruct the beam to obtain at least oneunobstructed portion of the beam of radiation. The second subsystem mayalso include an imager to image the at least one unobstructed portion ofthe beam of radiation to obtain a first set of electrical signals.

The processor may be further operable to process the first set ofelectrical signals to obtain the profile and features of the part to beidentified.

The part to be identified may be at least partially conductive orsemiconductive. A feature of at least one of the templates may includean eddy current signature. The system may further include an eddycurrent sensor to induce an eddy current in the rotating part and tosense the induced eddy current. The processor may also compare the eddycurrent signature with the sensed eddy current.

The part to be identified may partially obstruct the beam of radiationto obtain first and second unobstructed portions. The first and secondunobstructed portions of the beam of radiation may be imaged to obtainthe first set of electrical signals.

Each of the first and second unobstructed portions of the beam ofradiation may contain a magnitude of radiation which is representativeof a respective geometric dimension of the part to be identified.

The part to be identified may have threads.

At least one of the templates may include at least one feature relatedto the threads.

The part may be an externally threaded fastener.

Each part axis may be defined as being central to its part and parallelto its length.

Other technical advantages will be readily apparent to one skilled inthe art from the following figures, descriptions and claims. Moreover,while specific advantages have been enumerated, various embodiments mayinclude all, some or none of the enumerated advantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side perspective view of various threaded and non-threadedaerospace fasteners having various features and possible defects thatcan be extracted and measured using at least one embodiment of thepresent invention;

FIG. 2 is a top perspective schematic view of a system constructed inaccordance with at least one embodiment of the present invention andshowing various components and subsystems of the system as noted bynumbers in a legend;

FIG. 3 is top plan view, partially broken away, of various componentsand subsystems of the system of FIG. 2;

FIG. 4 is a top perspective schematic view, partially broken away, ofvarious components and subsystems shown in FIG. 3 with particularemphasis on optical inspection stations and removal mechanisms of FIG.3;

FIG. 5 is a top perspective schematic enlarged view, partially brokenaway, of one of the vision-based robotic subsystems and a returnconveyor of the system;

FIG. 6 is a view similar to the view of FIG. 5 but from a differentangle to particularly illustrate how a part exits from the inspectionstation and is received by a part inspection conveyor of the system;

FIG. 7 is an enlarged top perspective view, partially broken away, ofone of the inspection stations with a threaded bolt under inspection andparticularly showing a rotary catch to route fasteners into the partexit of the station;

FIG. 8 is a view similar to the view of FIG. 7, but taken underneath thestation and from a different angle to show previously inspected boltsexiting the station;

FIG. 9 is a side schematic view, partially broken away, of theinspection station of many of the previous figures and showing many ofthe components and subsystems of the station for complete dimensionalinspection and defect detection without the need for fixtures;

FIG. 10 is a top perspective view, partially broken away, of componentsof a fastener sorting subsystem wherein an identified and inspectedthreaded fastener is being pushed into a bin location of a bin assemblyby an electric cylinder;

FIG. 11 is a top plan view, partially broken away, of the subsystem ofFIG. 10 wherein a threaded fastener is about to slide down into the binlocation;

FIG. 12 is a side perspective view, partially broken away, of variouscomponents of the sorting subsystem and an electric cylinder with agripper in its retracted position used to slide out a floor of a binlocation containing fasteners in order to allow the fasteners in thatlocation to drop from the bin location onto a conveyor of a baggingsubsystem;

FIG. 13 is a view similar to the view of FIG. 12, but with the cylinderin its extended position in preparation to grip and slide out the floorof the bin location;

FIG. 14 is a top perspective view of one of the bin module assemblies ofthe sorting subsystem;

FIG. 15 is a bottom perspective view of the assembly of FIG. 14;

FIG. 16 is a view similar to the view of FIG. 14 from a different angle;

FIG. 17 is a top perspective view, partially broken away, illustratingthe part inspection conveyors and a bagger conveyor and bagging machineof the bagging subsystem;

FIGS. 18a and 18b are side schematic block diagram views, partiallybroken away, of various components and subsystems of a systemconstructed in accordance with at least one embodiment of the presentinvention including optical inspection devices and an actuator assembly;

FIG. 19 is a top plan schematic view, partially broken away, of variouscomponents and subsystems of FIG. 18a including a motorized rotarysubsystem, an eddy current sensor and a color camera;

FIG. 20 is a typical customer print specification by fastener family inthe form of a spreadsheet wherein various views of a typical threadedfastener are shown;

FIG. 21 is a view, partially broken away, of a database populated withdata after uploading from the spreadsheet of FIG. 20;

FIGS. 22a and 22b are schematic views of screen shots (with displayedicons and data) from a user interface of a master or system computercontroller; the screen shot of FIG. 22a shows a bolt in profile and thescreen shot of FIG. 22b shows a side schematic view of the bolt withregions of interest highlighted to search for visual defects;

FIG. 23 is a view similar to the view of FIG. 22a showing a shank of thefastener of FIGS. 22a and 22b with particular emphasis on its diameter;

FIG. 24 is a view similar to the view of FIG. 23 with particularemphasis on the length of the shank;

FIG. 25 is a view similar to the views of FIGS. 23 and 24 withparticular emphasis and the diameter of a head of the fastener;

FIG. 26 is a view similar to the views of FIGS. 23, 24 and 25 andpartially illustrating a 3 wire thread modeling algorithm (such as thethread model described in the '278 patent noted above);

FIG. 27 is a view similar to the views of FIGS. 23, 24, 25 and 26 withparticular emphasis on a thread anchor;

FIG. 28 is a view similar to the views of FIGS. 23, 24, 25, 26 and 27with particular emphasis on a thread major diameter; and

FIG. 29 is a block diagram flowchart illustrating a decision tree foridentifying a fastener in at least one embodiment of the presentinvention.

DETAILED DESCRIPTION

As required, detailed embodiments of the present invention are disclosedherein; however, it is to be understood that the disclosed embodimentsare merely exemplary of the invention that may be embodied in variousand alternative forms. The figures are not necessarily to scale; somefeatures may be exaggerated or minimized to show details of particularcomponents. Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to variouslyemploy the present invention.

Referring now to FIG. 2, there is illustrated an overview or systemlevel diagram which shows the various components and subsystems of atleast one embodiment of a system, generally indicated at 10, constructedin accordance with the present invention. Numbers 1 through 10 areindicated in a legend and are encircled on or near the variousidentified components and subsystems to facilitate understanding of thesystem and its corresponding method. The system 10 may be referred to asARIS (Automatic Reclamation and Inspection System): a self-containedmachine designed to automatically perform complete dimensional andvisual inspection of mixed parts such as aerospace fasteners 36 torecertify, reclassify, package and place them into inventory for futureuse.

Some of the many benefits of the use of the system 10 and itscorresponding method include but are not limited to: eliminate or reducewaste; time savings; improve quality; and positive impact on the bottomline.

In other words, the system 10 or machine is a technical system which canbe used to recertify aerospace fasteners 36 as well as self-supportingparts. Bolts, nuts, and collars are introduced into the system for thepurpose of part identification and certifying parts which fall into oneof two classes. The first class is new and unused parts. These parts arecapable of being re-introduced into inventory. The second class is partsthat are not fit to be used and must be removed. The system 10 includesmany devices and subsystems working together to create seamless,consistent, automated and accurate results.

Referring now to FIGS. 2 and 3, initially at an infeed or pre-feed stage(labeled “1” in FIG. 2), an operator dumps mixed fasteners 36 or partsinto a large, low profile vibratory hopper. Parts vibrate onto apre-screen subsystem (i.e. “2” in FIG. 2) that separates parts bygeneral size. This eliminates the opportunity for larger parts to ‘hide’smaller parts as will be evident hereinbelow.

Parts are then fed through a gentle-handling, vibratory carpet feedingsubsystem thereby transporting parts into multiple lanes. The pre-screensubsystem is a multi-lane vibratory v-shaped transport system that movesparts into single files, gently transitioning them onto a pick conveyor19 (i.e. “3” in FIG. 2) making a subsequent pick process by a robot 21more accurate and efficient.

The pick conveyor 19 is a multi-lane conveyor controlled by the mastercontroller (FIG. 18a ) via a conveyor controller 33 (FIG. 18b ) whichtransports and positions parts to two vision-guided robots 21 (i.e. “4”in FIG. 2) after traveling through the fields of view 30 of a pair ofcameras 32. The cameras 32 and the robots 21 are mounted on supportbeams of a support frame structure 34 of the system 10. A rough texturedbelt of the pick conveyor 19 minimizes part movement during transport.

The vision-guided robots 21 have the ability to pick up any part withina specified range of allowable parts using multiple end-of-arm toolingor grippers 17. The robots 21 pick up the bolts, nuts and collars andorient them at two inspection stations, each of which is referred to asa VisionLab. Each of the parts has a part axis and each of theinspection stations has a measurement axis 13 (i.e. FIG. 18a ). Eachrobot 21 precisely positions self-supporting fasteners 36 on afixtureless, rotary support or stage, generally indicated at 14, at itsinspection station so that the part and measurement axes aresubstantially centered and aligned.

The robots 21 are preferably six axis robots located adjacent the pickconveyor 19 and at the VisionLab inspection stations. Each robot 21 isvision-guided to identify, pick, orient, and present the parts 36 “headdown” so that they are self-supporting on the VisionLab glass stage 14.The grippers 17 accommodate multiple part families.

Benefits of Vision-based Robot Automation include but are not limited tothe following:

Smooth motion in high speed applications;

Handles multiple parts 36;

Slim designs to operate in narrow spaces;

Integrated vision; and

Dual end-of-arm tooling or grippers 17 designed to handle multiple partfamilies.

A master control station or system controller (FIGS. 18a and “10” inFIG. 2) determines locations and orientations of the fasteners 36 on thepick conveyor 19 using any suitable machine vision system having atleast one camera (i.e. camera 32). Any one or more of variousarrangements of vision systems may be used for providing visualinformation from image processors (FIG. 18b ) to the master controller.In one example shown in FIGS. 4, 5, and 6, the vision system includestwo three-dimensional stationary cameras 32 that provides light overfields of vision or view 30, creating a stripe of light (or otherpattern) across the fasteners 36 as they pass under the cameras 32 onthe conveyor belt of the conveyor 19. In various embodiments, the lightmay be a laser beam. The cameras 32, their image processors and themaster controller may be configured to locate various features such asholes or heads or threads of the fasteners 36. Alternatively, or inaddition, the master controller may register the contours of thefasteners based on the various depths of the light on the surfaces ofthe fasteners 36.

In some embodiments, multiple cameras such as the cameras 32 can besituated at fixed locations on the frame structure 34 at the inspectionstations, or may be mounted on the arms of the robot 21. FIG. 4 showstwo cameras 32 spaced apart from one another on the frame structure 34.The cameras 32 are operatively connected to the master controller viatheir respective image processors. The master controller also controlsthe robots 21 of the system 10 through their respective robotcontrollers 23 (FIG. 18b ). Based on the information received from thecameras 32, the master controller then provides control signals to therobot controllers 23 that actuate robotic arm(s) of the one or morerobot(s) 21 used in the method and system.

The master controller at the master control station can include aprocessor and a memory on which is recorded instructions or code forcommunicating with the robot controllers 23, the vision systems, therobotic system sensor(s), etc. The master controller is configured toexecute the instructions from its memory, via its processor. Forexample, the master controller can be a host machine or distributedsystem, e.g., a computer such as a digital computer or microcomputer,acting as a control module having a processor and, as the memory,tangible, non-transitory computer-readable memory such as read-onlymemory (ROM) or flash memory. The master controller can also have randomaccess memory (RAM), electrically-erasable, programmable, read onlymemory (EEPROM), a high-speed clock, analog-to-digital (A/D) and/ordigital-to-analog (D/A) circuitry, and any required input/outputcircuitry and associated devices, as well as any required signalconditioning and/or signal buffering circuitry. Therefore, the mastercontroller can include all software, hardware, memory, algorithms,connections, sensors, etc., necessary to monitor and control the visionsubsystem, the robotic subsystem, etc. As such, a control method can beembodied as software or firmware associated with the master controller.It is to be appreciated that the master controller can also include anydevice capable of analyzing data from various sensors, comparing data,making the necessary decisions required to control and monitor thevision subsystem, the robotic subsystem, sensors. etc.

An end effector on the robot arm may include a series of grippers 17supported to pick up the fasteners 36. The robotic arm is then actuatedby its controller 23 to pick up the fasteners 36 with the particulargripper 17 from the conveyor 19, positioning the gripper 17 relative tothe fasteners 36 using the determined location from the visual positiondata of the particular vision subsystem including its camera 32 andimage processor (FIG. 18b ).

In general, each VisionLab inspection subsystem is a inspectionsubsystem equipped with a rotating glass stage 14. Vision Lab inspectionincludes 360 degrees of coverage for dimensional and visual defectpurposes. Each VisionLab subsystem identifies and inspects the givenself-supporting part/fastener 36 based on criteria specific to the part.

Each VisionLab may have:

Side, top and bottom view high resolution cameras, lenses (some of whichare electronically-controlled liquid lenses having a variable focallength so that fasteners of various sizes and shapes can be identifiedvia inspection) and lighting (bottom view through VisionLab's rotatingsapphire glass support or stage 14);

Color vision;

Sequenced back and front lighting for 360° dimensional and visual defectinspection;

Internal view of some fasteners 36; and

Smart motor and software combined to provide measurements and visualinspection every 1 degree of rotary motion for a total of 720 inspectionpoints.

Referring now to FIGS. 4 through 9, Each VisionLab typically includes ahigh-resolution camera 304 with collimated lighting from a lightingsource 300 and a custom telecentric lens 302 for complete dimensionalinspection plus optional sequenced front LED lighting 350 for visualdefect detection. Other options include bottom vision with the bottomlighting 200 for head markings and surface inspection, a top camera 310with liquid lens for drive/recess inspection, an internal threadinspection camera (not shown) and liquid lens and a sensor head or colorcamera 400 and liquid lens for color vision.

Capabilities of the VisionLab station or subsystem (i.e. ARIS InspectionStation) include:

Side-View Dimensional Characteristics

Diameters

Threads

Tapers

Lengths

Concentricity

Straightness

Radii

Length from specified diameter (head protrusion)

Diameter from specified length position

Parallelism/Perpendicularity

TIR

Theoretical Intersection

Formula

Surface Characteristics

Seams

Cracks

Visual Defects

Internal Thread Inspection (with the liquid lens technology)

Color Camera (with the Liquid Lens Technology)

Plating Inspection, color verification

Top and Bottom-View Inspection (Top View with the Liquid LensTechnology)

Head Stamp Verification

Visual Defects (bottom surface)

Head Diameter

Head Roundness

Drive/Recess Inspection

Referring now to FIGS. 18a, 18b and 19 (FIGS. 9 and 10 of U.S. Pat. No.10,207,297), as described in U.S. Pat. No. 10,207,297, each of theinspection systems has a measurement axis 13. A set of opticalinspection devices of a first embodiment of the system is illustrated inFIGS. 2, 3, 9 and 10 of the '297 patent. Optical inspection devices ofthe system 10 typically include a high-speed, high resolution camera, alens, an optical depth sensor (such as a triangulation-based sensor),back lighting, front and/or top lighting or illuminator, a top camera,bottom lighting or illuminator and a bottom camera.

Example self-standing parts, such as threaded parts with ball-shaped endportions and a threaded fastener or bolt, has threads, a length betweenits ends, a width, and a part axis which, preferably, is central to thepart and parallel to its length. A variety of manufactured parts whichmay be inspected are shown in the drawing figures, including FIG. 1 ofthe '297 patent. In one example embodiment, the parts may have a lengthof 10″, a diameter of 2″, a length repeatability of 10 microns, adiameter repeatability of 2 microns, and an inspection speed of about 7parts per minute. Also, typically the self-standing parts are capable ofrotating without falling over (i.e. without the need for a partfixture).

The threaded bolt 36 is supported on a transparent (i.e. scratch-proofglass, plastic, or sapphire) window 11 of the motorized rotary orrotation fixtureless stage 14. The part is able to stand on and rotatewith the window 11 without falling over (i.e. the part is capable ofstaying up or upright) without the need for a part fixture (i.e., thepart is self-supporting).

As shown in FIG. 18A, a weight sensor 16, preferably comprising a loadcell, may be coupled at the lower surface of the window 11 to measureweight of the part supported on the window 16 to help identify the part.The load cell is electrically connected to the system controller toprovide a signal which represents the weight of the part.

The weight sensor 16 may be implemented as a strain gauge including astrain gauge load cell and resistive wire connected to a load platecoupled to the bottom surface of the window 11. The load plate and thewindow 11 typically displace a small distance corresponding to the partweight.

The weight sensor 16 alternatively may be a piezo-based weight sensor, aQTC sensor, a capacitance-based weight sensor or an inductance-basedweight sensor to measure microgram or milligram changes in weight causedby the weight of the part on the top surface of the window 11.

Because the part is self-supporting on the transparent window 11 of thestage 14 without the need for fixtures or other devices, top and bottomcameras and corresponding strobed illuminating LED ring lights areprovided to obtain top and bottom end views of the part, respectively.The rotation stage may be a precision rotation stage having relativelylarge (i.e. 100 mm) central aperture over which the transparent windowis fixedly secured to rotate with an annular plate. An encoder 15 (i.e.,FIG. 18a ) provides an output signal based on the amount of rotarymovement of the window. Such rotary stages are available from PI micosand may utilize a DC servo motor with a rotary encoder on a rotary shaftof the motor or a stepper motor. A worm drive with a high gear ratio maybe provided between the shaft and the annular late for precision angularpositioning of the transparent window, and, consequently, the part.

The bottom vision subsystem includes the bottom lighting and the bottomcamera 210 both of which are located below the glass window as shown inFIGS. 2 and 9 of the '297 patent at a bottom imaging location. Thecamera 210 is preferably a single view camera with image analysissoftware that minimizes surface and lighting variations. The lightingilluminates the part.

Detects which the bottom vision subsystem can detect with respect to thepart:

Min thru-hole;

Cracks on flange;

Functional OD Hex;

Cracks on ball portion;

Flange side ID cracks; and

Top and bottom ID crack.

The illuminator and the lens detector subsystem (i.e. camera) may bereplaced by a high-speed 2D/3D laser scanner available form KeyenceCorporation of Japan.

The rotary part stage 14 typically includes an electric actuator ormotor which may be stepper motor or a DC motor.

The parts of FIG. 1 of the '297 patent can be rotatably driven by themotor via motor-driver or controller via system controller while beingsupported vertically (i.e., the parts are self-supporting) as shown inFIG. 9 of the '297 patent.

The various components or functions of the motor driver or controller ofFIG. 9 of the '297 patent may be implemented by the separate motorcontroller as illustrated, or may be integrated or incorporated into thesystem controller, or other controller, depending upon the particularapplication and implementation. The MCU (i.e. motor control unit)typically include the control logic to control the rotary stage 14. Thecontrol logic may be implemented in hardware, software, or a combinationof hardware and software.

One or more memory devices within the system controller and/or the motorcontroller may store a plurality of activation schemes for the rotatablewindow 11 and may represent any one or more of a number of knownprocessing strategies such as event-driven, interrupt-driven,multi-tasking, multi-threading, and the like. As such, various steps orfunctions may be performed in sequence, in a modified sequence, inparallel, or in some cases omitted. Likewise, the order or operation orprocessing is not necessary required to achieve the objects, features,and advantages of the invention, but is provided for ease ofillustration and description.

Preferably, the control logic is implemented primarily in softwareexecuted by a microprocessor-based controller or the microcontroller(i.e. MCU). Of course, the control logic may be implemented in software,hardware, or a combination of software and hardware depending upon theparticular application. When implemented in software, the control logicis preferably provided in a computer-readable storage medium havingstored data representing instructions executed by a computer to controlangular position and rotation of the window 11 of the system 10 throughthe rotary stage 14. The computer-readable storage medium or media maybe any of a number of known physical devices which utilize electric,magnetic, and/or optical devices to temporarily or persistently storeexecutable instructions and associated calibration information,operating variables, and the like.

In one example embodiment, the stage 14 is electromechanically driven bya rotary actuator such as the DC motor 28 and associated transmission(not shown) in the form of a worm gear or the like. The DC motor may be,for example, a brushed or brushless DC servomotor, the operation ofwhich is controlled by the motor controller via a motor drive or driverwithin the motor controller. The brushed or brushless motor may have itsrotary speed and position controlled by pulse width modulation (PWM)control.

The motor controller outputs motor drive commands to the DC motor basedon outputs from the rotary encoder 15 and/or decoded commands from thesystem controller. The motor controller controls the DC motor throughthe motor drive of the motor controller so that the angular position ofthe stage 14 is changed. In other words, the system controller outputsservomotor drive commands to the motor controller which controls the DCmotor and, through its transmission, the stage 14.

As described in U.S. Pat. No. 8,550,444 (also owned by the assignee ofthe present application), the system 10 may include a part-centering andaligning subsystem. The subsystem or apparatus ensures that a part iscentered in the system 10 and that the part is aligned with themeasurement axis (Z-axis) 13 without the need to measure any distancesor angles. In other words, the apparatus ensures that the part isproperly placed or positioned on the window 11 in the system 10.

As described in U.S. Pat. No. 8,550,444, the part-centering apparatustypically includes a carrier which defines a part receiving cavity. Theapparatus also has a central axis substantially parallel to themeasurement axis 13 or Z-axis and includes a plurality of members orlevers having open and closed positions. The members having holdingfaces which are substantially equidistant from the central axis duringmovement between the open and closed positions. At least one of themembers applies a force on an exterior side surface of a part, disposedbetween the holding faces during movement between the positions toreposition the part. The repositioned part is centered and aligned withrespect to the measurement axis 13. The holding faces releasably holdthe repositioned part in the holding position between the open andclosed positions of the members.

The part-centering apparatus may also typically include automaticallyoperable lever arms which are coupled to their respective relativelymoveable, spring-biased ring members of the carrier. Movement of one ofthe lever arms either towards or away from the other lever arm(depending on the biasing of the spring(s)) causes the members to movefrom their open position to their holding position against the part tocenter and align the part.

The system 10 may also include a moveable stage subsystem coupled to thepart-centering apparatus for sliding the apparatus relative to therepositioned part along the central or measurement axis in the openposition of the members to allow the exterior side surface of therepositioned part to be measured. In turn, the slide/base unit moves themoveable stage subsystem up and down. A horizontal support membercouples the subsystem to the apparatus to move the apparatus along thecentral or measurement axis 13.

The system 10 may further include a mechanism which is coupled to oneend of the support member for translating the support member and theapparatus a limited extent relative to the subsystem along the centralaxis.

Referring again to FIGS. 3, 9 and 10 of the '297 patent, the system alsoincludes a backside illumination assembly, generally included at 300. Ingeneral, back lighting provides measurement of profile characteristics.This provides maximum, minimum or average measurements, simultaneouslyor separately for features like: Radii Concentricity, Straightness,Lengths, Diameters and Threads for threaded parts.

The illumination assembly 300 directs a beam of collimated radiation atsubstantially the entire backside surface of the self-supporting part 36at predetermined angular increments of movement of the part about themeasurement axis 13 of the system 10 during a rotational scan. The beamis occluded by the self-supporting part at each increment of movement tocreate a stream of unobstructed portions of the beam in rapid successionpassing by and not blocked by the self-supporting part.

Preferably, substantially the entire backside surface is completelyenclosed by a beam profile of the beam. The beam profile is generallyrectangular with a height greater than or equal to the length of thepart and a width greater than or equal to the width of the part as shownin FIGS. 18 and 19.

The assembly 300 may be moveable up or down via a motor driven orcontrolled by a driver/controller upon receiving a control signal fromthe system controller as shown in U.S. Pat. No. 9,370,799.

The illumination assembly or radiant source 300 illuminates an objectsuch as a threaded bolt to be imaged, and a telecentric optical lens 302(i.e. FIGS. 2, 3, 9 and 10 of the '297 patent) receives the radiationpassing by and not blocked by the part and guides it towards an imageplane of the image acquisition device or detector, generally referred as304. Consequently, the radiation source 300 preferably comprises a LEDemitter including a plurality of LED emitter elements serving to emitradiation in either the visible or ultraviolet range. The LED emitter ofthe source 300 is preferably high power, capable of generating 100optical mW or more for each emitting element. A lens (not shown)collimates the radiation.

As shown in U.S. Pat. No. 9,370,799, the back light 300 and the detector304 may be coupled together by a yoke to rotate together about the part36 via a motor via a driver/controller upon receiving a command signalfrom the system controller.

An optical or optoelectronic device for the acquisition of images (forexample the camera or telecamera 304) has the image plane which can be,for example, an electronic sensor (CCD, CMOS). The self-supportingfastener, bolt or other manufactured part, is received on an retained ata position and orientation for optical inspection by the fixturelesstransparent glass or plastic of the window 11 of the system 10.Preferably, the device 304 is a high speed, high resolution digitaltelecamera, having an electronic sensor with individual pixels oflateral dimensions equal to or less than one or more microns.

As shown in U.S. Pat. No. 9,370,799, the assembly 304 can be driven upand/or down by a motor via a driver/controller upon receiving anappropriate control signal from the system controller. Typically,movement of the assembly 304 and the backlight 300 is coordinated by thesystem controller so that they move in unison.

As described in U.S. Pat. No. 9,370,799, the lens 302 typicallycomprises a forward set of optical elements proximal to the manufacturedpart, a rear optical element proximal to the acquisition device and anaperture diaphragm interposed between the forward and rear sets ofoptical elements. The aperture diaphragm comprises a circular windowtransparent to the radiation, which is referred to as a diaphragmaperture. For example, the aperture diaphragm can comprise an opaqueplate preferably of thickness of a few tenths of a millimeter, and thediaphragm aperture can be defined a simple hole in the plate.

The diaphragm aperture or window is coaxial to the optical axis of theforward set of optical elements, and positioned on the focal plane ofthe forward set defined for the wavelength range of radiation emitted bythe radiant source.

The position of the focal plane of a set of optical elements mostlydepends on the refraction index of the material from which the lensesare made, which, in turn, depends on the wavelength of theelectromagnetic radiation passing through the lenses.

The lenses 302 only accepts ray cones exhibiting a main (barycentric)axis that is parallel to the optical axis of the forward set. Therebythe lens 302 is a telecentric lens configured for the particularradiation. The rear set of optical elements serves to compensate andcorrect the residual chromatic dispersion generated by the forward setoptical elements for the wavelength in question.

The optical axis of the rear set coincides with the optical axis of theforward set and the focal plane of the rear set defined for thewavelength cited above, coincides with the plane on which the aperturediaphragm is located. Consequently, rays of radiation conveyed by therear set towards the image lane form light cones, the main (barycentric)axis of which is parallel to the optical axis of the lens 302.

The forward set preferably includes two positive lenses, which canexhibit a flat-convex, bi-convex, or meniscus shape. The positive lensescan both be made in common optical glass. For example, they can both bemade in low chromatic dispersion crown glass.

The rear set of optical elements preferably comprises four lenses. Thelens which is proximal to the diaphragm can be a negative lens servingto partially or completely correct the chromatic aberrations generatedby the forward set. The negative lens can be bi-concave, flat-concave,or meniscus shaped, and can be made of common optical glass. Forexample, it can be made of high chromatic dispersion flint glass.

The three rear lenses are positive lenses that can all be made ofoptical glass, for example, in low chromatic dispersion crown glass.

The lens 302 is therefore both telecentric on the object side andtelecentric on the image side, and overall the lens 302 is abi-telecentric lens configured for light such as visible light orultraviolet light.

Dimensional features that can be measured via the above-described sidevision devices include:

Diameters;

Tapers

Lengths

Concentricity

Straightness

Parallelism

Perpendicularity

Threads

Pitch

Major Dia

Pitch Dia

Referring now to FIGS. 9 and 10 of the '297 patent, there is illustrateda triangulation-based sensor head, generally indicated at 400.Alternatively, the device 400 may comprise a color camera having aliquid lens. The offset color camera 400 preferably utilizes liquid lenstechnology and advanced image analysis software that minimizes surfaceand lighting variations. A strobe LED ring light illuminates the outerdiameter surface for dedicated coating/color inspection. The liquid lenstechnology allows the camera to stay stationary and not move because theeffective focal length of the camera lens can be electronicallycontrolled. Color inspection is thus provided for proper fastenercoating, color and quality.

The sensor head 400 may comprise a high-speed, 2D/3D laser scanneravailable from Keyence Corporation of Japan. Such a sensor head fromKeyence generates a laser beam that has been expanded into a line and isreflected from the side surface of the part as well as any radiallyextending surfaces of the part, such as the threaded bolt 36. Thereflected line of light is formed on a sensor, and by detecting changesformed on a sensor, and by detecting changes in the position and shapeof the reflection, it is possible to measure the position of variouspoints along the surface of the part.

Alternatively, during the scans of the side profile, a laser line may bepainted on the part and a vision subsystem positioned on either side ofthe laser line receives reflected laser light and the resulting imagesprovides a 3D image (including Z axis or depth). In this way, bothvisual defects and measurement features or characteristics that requirea depth component are simultaneously extracted.

Alternatively, (not shown herein but shown in U.S. Pat. No. 9,370,799the entire disclosure of which is hereby incorporated by referenceherein), the sensor head 400 may rotate and/or linearly move via a motorvia a rotary driver/controller and/or a linear driver/controller,respectively, upon receiving command signals from the system controller.A transmission may convert the rotary motion of the motor output shaftto linear motion.

As the manufactured part rotates, corresponding sets of 2D profilesignals are generated by the sensor head 400. At least one processorprocesses the sets of 2D profile signals to obtain a 3D view of thecomplete side and any radially extending surfaces of the part.

The system controller provides control signals based on the signals fromthe rotary sensor or encoder 15. Alternatively, sensor(s) and/orencoder(s) are not required if stepper motor(s) are provided.Alternatively, or additionally, the signals from the rotary encoder 15are directly utilized by the sensor head 400 at the station to controlthe sensor head 400. The control signals are utilized to control thesensor head 400 which preferably have encoder inputs which allow precisecontrol over the position of 2D profile signals samples.

At least one signal processor may process the sets of 2D profile signalsto identify a defective part as described in greater detail in U.S. Pat.No. 9,370,799, also owned by the assignee of the present application.The at least one processor may process the sets of 2D profile signals toobtain one or more measurements of the part.

The operator may tell the system controller via a display 500 and userinterface 502 (i.e. FIG. 18a ) where the interesting parameters arelocated on the Z axis (height of the part). Then, the software toolsextract and measure features from the images and resulting 2D profilesignals created by the reflected lines of radiation.

The 2D profile signals may be processed by the at least one processorunder system control to obtain a 360 degree panorama composite view orimage which is used by the processor to determine at least one of adent, a split, a perforation, a crack, a scratch, a wrinkle, a buckle,or bulge, and a surface blemish located at the side surfaces of the partwhere the part is an ammunition case or a fastener 36.

A top vision subsystem typically includes the frontside illuminationdevice which may include the strobed ring LED illuminator 350. Theilluminator 350 typically includes a curved array of LED light sources,groups of which are under control of the system controller to providedirect illumination of the front of the case or fastener 36 and are usedto enhance defects in the front surface of the fastener or bolt 36.Alternatively, the frontside illumination device may be side-mounted sothat the front light comes from the side of the part and not from abovethe part, i.e., basically like painting a thin line along the length ofthe part.

The top vision subsystem also typically includes a single view camerawith image analysis software that minimizes surface and lightingvariations. A lens of the camera may also utilize liquid lens technologyas described above. Lighting which illuminates the part includessoftware. The lens of the camera may be up to 2 inches in diameter. Thetop vision subsystem may detect such defects as castle chip out;functional ID Hex; and top and bottom ID cracks. Front lighting providessurface defect detection for tool chatter, cracks and other surfaceimperfections.

The detected optical images are processed by the image processor todetermine at least one of a dent, a split, a perforation, a crack, ascratch, a wrinkle, a buckle, a bulge, and a surface blemish located atthe side surfaces of the fastener.

Referring to FIGS. 18A and 19, there is illustrated an eddy currentsensor which includes coils (not shown) which not only induce an eddycurrent in the rotating part but also senses the induced eddy current toprovide a signal to eddy current electronics which represents the amountof induced eddy current. Typically, the sensed eddy current is comparedwith an eddy current signature of a “good” part. The eddy current sensorand electronics can be used to inspect for various metallurgical defectssuch as seams, cracks, porosity, heat treat variations and conductiveplating variations.

Referring to FIG. 19, the system may include a commercially availableX-ray fluorescence (XRF) analyzer to determine the composition of ametal alloy of the part when the part is supported on the window 11.Typically, X-ray fluorescence is detected or sensed by a detectorconnected to electronics which, in turn, may provide a signal which isconverted into a signature that is subsequently compared to signaturesof reference materials in order to identify and/or classify the materialof the part.

Data/Image Processor for the Detection of Surface Defects on SmallManufactured Parts

The vision system is especially designed for the inspection ofrelatively small manufactured parts (i.e. especially the parts ofFIG. 1) which typically have a diameter of 1 mm to 50 mm and a length ofup to 228 mm. The processing of images of the cartridge cases and likeparts to detect defective cases is generally described in issued U.S.Pat. No. 7,403,872 which also describes the processing of sensed inducededdy current.

At least one embodiment of the present invention utilizes many of theteachings found in U.S. Pat. No. 7,633,635 ('635 patent), assigned tothe assignee of the present application to automatically identifyunidentified mixed fasteners.

This embodiment (of the '635 patent) has a part setup procedure for auser to capture an image of a known dimensioned part and define a set offeatures with acceptable range of limit values for them. The partprofile and features are referred to as the part template. During partID mode, the profile of each inspected part is captured and its featuresare compared to their limit values. If any feature of a part isdetermined to be outside its range of limits, then it is not identified.

Referring to FIG. 1 of the '635 patent, there is illustrated oneembodiment of a system, generally indicated at 20, for automaticallyidentifying non-identified fasteners. Such fasteners may include, asillustrated in FIG. 1, fasteners, whether threaded or not, such as nuts,bolts, nails. Such fasteners may include flat parts such as washers orcylindrical or near-cylindrical (i.e., have a small cosine error) partssuch as plastic tubular members. The fasteners may be at least partiallyconductive, semiconductive, or conductive. The fasteners may be platedor non-plated, heat-treated or non-heat-treaded, or include seams.Typically, the fasteners may have a diameter range of 2 mm to 35 mm anda length range of 10 mm to 150 mm.

The data and signal processing system described therein (i.e. the '635patent) illustrates how the system processes sensor data and discoversthe ID of the fastener presented to the system. Using calibration data,sensor data is transformed to a description of the outline of thefastener, specified in calibrated physical coordinates. Featureprocessing extracts values for each feature contained in the entire parttemplate data set. Match metric processing identifies the best match tothe sensor data among the fastener templates. ID generation evaluatesthe best match; if the match is good enough, the part is said to beidentified, otherwise the part is not identified. After ID generation, amessage is sent to the master computer containing the part ID or a “notmatched” indication.

When a new part is added to the system 20, a file called a “template” isautomatically created as described in detail hereinbelow. The templatefile contains information about the fastener that is used to identifyit. The template is set up so that any fastener of the given type willmatch the template, and any fastener not of the given type will notmatch.

When a robot 21 of the present application places a self-supportingfastener to be identified on its fixtureless stage 14, the softwareacquires data containing the profile and eddy signature of the fastener.The software then checks all the templates in the list to find a match.If the fastener matches one of the templates, then the fastener isidentified. If none of the templates match, the fastener is notidentified.

In general, when setting up a new fastener, the user chooses “features”of the part to be measured. The measurements of the features willdistinguish the new fastener from the other fasteners in the system. Thetypes of features include total length, internal length, diameter,thread, taper, and eddy current signature. For most features, the userchooses a region of the fastener where the measurement will be made, anominal value of the measurement (the value the fastener should have ifit's the right fastener, and plus and minus tolerances which determineif the measurement is close enough to match the fastener. For somefeatures, such as total length and eddy current, the measurement regionis the whole fastener. Also, for eddy current the user chooses arectangular on the eddy screen of a display instead of a nominal valueand tolerances. If the eddy signature hits the rectangle, then thefastener is a match.

The user chooses which features are needed to distinguish the newfastener. For a wire nut, for example, the user would typically add atotal length feature and a taper feature. A bolt may need total length,thread, and one or two diameters. If it is necessary to distinguish thetype of material or coating to distinguish a bolt from one another bolt,the user would add the eddy feature.

When all of the necessary features have been set up automatically in atemplate, the user saves the template. This is added to the list oftemplates to check when a fastener is in the part identificationoperation, as previously described.

More particularly, in automatically creating a template a gold or masterpart with known good dimensions is not typically used. An image of thefastener is displayed on a screen, as generally shown in the screen shotof FIG. 6 of the '635 patent.

After a good image of the part is automatically obtained, featuresautomatically are added to the template as previously mentioned. Forexample, when adding an internal length, points are determined on thefastener when one wishes to measure the internal length (i.e., here thelength of the head of the bolt). One can add multiple internal lengthsfor each fastener. Internal lengths can be use to measure features like:thread length, shoulder length, head height, under the head to the startof a part, and any length measurement needed inside of a fastener.

Such predefined points are also useful for other template features likediameters and tapers. Such predefined points are useful when looking forrising and falling edges of the fastener as well as when looking forminimum and maximum diameters of the fastener.

The diameter feature is used to measure diameters around the fastener.Multiple diameters can be added for each fastener. One can selectminimum and maximum diameters for a selected area (or a small groovewithin a selected area) or one can average all the diameters in the areaselected.

With respect to taper features, tapers are used to measure taperedangles on a fastener. Multiple tapers can be added for each fastener.

The external/overall length feature is automatically added to the listof features. With respect to the thread features, the tolerances on thefollowing thread features can change: thread count, thread pitch, pitchdiameter, functional size, lead deviation, minor diameter, and majordiameter.

With respect to the code, the designated code can be entered on a touchscreen to identify the fastener corresponding to a particular template,which also shows various features of a bolt.

With respect to eddy current, a frequency parameter is initially set upfor a particular fastener. A relatively low frequency such as 1 KHz maybe used to check for material and a relatively high frequency such as 50KHz may be used to check for plating of a fastener. A template for eddycurrent is automatically generated. After obtaining a signature, one mayhave to adjust the parameters of the frequencies and the gains whiletesting a good fastener, until a good image is obtained on the screen ofa display. A good image should have a defined area, like a loop, thatwill have some space inside it. After establishing the eddy currentsignature of a good fastener, the area of the signature one wants toinspect may be highlighted.

Referring now to FIGS. 6, 7, 8 and 10-17 of the present application,after possible fastener identification and inspection, the inspectedfastener will be pushed off the VisionLab stage 14 via an electriccylinder 40 onto a rotary catch 42 that will route the part into a queuestation and then onto a transport device of a precision linear slide,generally indicated at 46 (i.e. “6” in FIG. 2).

There are two linear slides or post inspection conveyors 44, one foreach VisionLab output. These slides 44 receive the parts intobasket-like, transparent devices 45. Each part is transported to one ofa plurality (i.e. here 40) of bin locations 49. Once a transport device45 arrives at the proper bin location 49, an electric part pushercylinder 46 pushes the part out of the basket-like device 45 and intoone of six locations 49 of a bin module assembly 48 (“7” in FIG. 2).Once the part is removed, the transport device 45 of the slide 44 (“6”in FIG. 2) returns to the VisionLab (“5” in FIG. 2) to pick up the nextpart.

Good Part Transport to Holding Bin Location of Bin Assembly

The high speed linear slides 44 transport accepted parts in holding cupsor devices 45 that the parts fall into after the VisionLab inspection.One high-speed slide 44 is dedicated to each VisionLab. Each transportcup 45 takes each accepted part to the assigned bin 49 of one of the binmodule assemblies 48 and the cylinder 40 pushes the part, allowing thepart to transition into a particular bin location 49 within thedesignated assembly 48. The transport device 45 then returns via itsslide 214 to retrieve the next accepted part from its VisionLab.

Reject Chute

Bad parts are sorted out to a reject bin (not shown). Rejected partswill drop into the first “bin” location which is designated for failparts only “containment bin.”

Each of the good bin locations 49 will know what part number andquantity reside in that bin location 49 via the master controller. Oncethe pre-determined quantity of parts has been met at a particular binlocation 49, the floor 51 of that bin location 49 will be slid out by anelectronic cylinder 51 having a gripper allowing the parts from that onebin location 49 to drop onto a bagging conveyor 53 (i.e. “8” in FIG. 2).

The bagging conveyor 53 then transports the parts to a bagger machine ofa bagging system 52 (i.e. “9” in FIG. 2) and the bagging machineexecutes packaging and labeling functions under control of the mastercontroller.

Good Part Holding Station/Bins

Once the parts are removed from its bin location 49 and transported tothe bagging machine, the bin location 49 becomes available for the nextsuccessfully passed part to occupy.

Conveyor System Transfers Parts to Bagger Machine

Full bins 49 are opened to the bottom bagging conveyor 53. Once a binlocation 49 is emptied, it is assigned a new part number.

Bagging System

Good parts enter the bagging system 52 from the transfer or baggingconveyor 53.

The bagging system 52 prints labels with part number and lotinformation, places the labels on the bags and finishes with sealing.

Referring now to FIGS. 20-28, there is illustrated an automatic templategeneration method and system as follows:

1. Customer Provides Part Print Data in a pre-defined Spreadsheet Formatof FIG. 20 (Features, Dimensions, Attributes and Tolerances). This isdone automatically using an algorithm which pulls the data that thecustomer has developed.

2. Spreadsheet Data Uploaded in to a VisionLab Database of FIG. 21

An import utility takes the spreadsheet data and converts it into aformat that the database can use.

3. Database data is transferred into VisionLab Fastener InspectionTemplates by Part Number

A physical fastener from a family of fasteners can be utilized to“train” and set up the template. The software uses the information inthe database to take the “trained” regions and automatically grow orshrunk them depending on the size of the fastener.

Fastener templates created through the spreadsheet data transfer processdescribed above include:

1. Part Number

2. All Selected Features

3. All Dimensions and Tolerances

4. Visual Defect Detection Regions of Interest and Level Settings

As noted in FIGS. 22a through 28, such selected features may includeshank diameter, shank lengths, and head diameter and various threadfastener features such as thread lengths, major and minor diameters, andpitch diameter.

As further described, the spreadsheet is created and provides all thedata needed to identify the part including: Part number, Part Family,Measurements and Attributes. Software is used to read each line of thespreadsheet and populate the information into objects. The data isdynamically read as there can multiple options for each bolt, nut andcollar that change the finish of the part or modify the measurements.

After all the data is read from the spreadsheet, it is populated intothe database using 4 normalized tables. The first table stores eachfamily type with the date it was added into the system and the type ofpart that family provides (bolt, nut, collar, sleeve, etc.). The secondtable stores the full part number, and each value of the part numberbroken down. Lastly, the data for each part is stored between twotables. One table stores the measurement values as minimums, nominals,maxes, ranges and the other table stores attribute values such astrue/false, colors and string values that match items such as headtypes.

Lastly, using the data an overlap report is generated. The overlapreport sorts all the part objects and compares the measurement values tofind where all the measurements are an exact match.

General Machine Overview

As previously described, a vibrating feed table divides the parts bypart height into 3 lanes. The parts are vibrated down each lane onto aslow-moving conveyor. Parts go through an optical sensor such as acamera that allows robots to identify the orientation and length of thepart. The robots will then move into position to pick up the part. Thereare 3 grippers on each robot that allow them to pick up a range of partsizes. The robot will move with the correct gripper to grab the part andplace it onto the center of one of the Vision Labs. Each robot has itsown Vision Lab. The Vision Lab stage will then spin and images of thepart are collected to identify it. After the Vision Lab has completedits cycle and has all its results, it will push the part off the tableor stage and spin to dump the part down a chute. After the part falls itwill be added into the queue station. The queue station can hold onepart.

A cup attached to a slide will collect the part from the queue anddeliver it to the appropriate bin location. Each bin location will holdone type of part. When the cup gets to the correct bin it will open toallow all the parts to fall into the bin. There are 42 bins in total: 1bin for failed parts, 1 bin for dumped parts, and 40 bins for goodidentified parts.

When bins get full, or the number of bins runs out, the fullest bin isselected according to their max part count, another slide will go to theappropriate bin and pull a trap door to release the parts onto a lowerconveyor. The parts will be pushed from the conveyor to a bagger whichwill be printed with a label. The label will contain the partinformation, a timestamp and how many parts are in the bag. The baggerwill cycle and drop the bag onto a final conveyor which will take therecertified parts away.

Referring now to FIG. 29, there is illustrated a block diagram flowchart which shows the part/fastener identification process/method of atleast one embodiment of the present invention.

At block 50, a top part is found that is closest to a number ofmeasurements.

At block 52, if the part fails the “foreign” part check (i.e. the partdoes not have a corresponding template), the part is dumped at block 54.If the part doesn't fail the check, block 56 is entered.

At block 56, the question is asked if this is the only part found thatmatches the measurements. If yes, the part is identified at block 58. Ifnot, block 60 is entered.

At block 60, the list of matching measurement parts is narrowed downusing identified attributes.

At block 62, if there is only one part remaining, the part is identifiedat block 64. If not, the part is dumped at block 66.

In summary, the system 10 includes in each VisionLab inspection station:

Side Vision Profile Dimensional Inspection and Visual Defect Detection

Bottom Vision with Front LED Lighting

Top Vision with Front LED Lighting

InternalView with Front LED Lighting

Color Vision with Front LED Lighting

MultiView Back Lighting: At the heart of VisionLab is the precisegauging side vision station that combines a high resolution CCD camera,telocentric lens, collimated back lighting with a smart motor andcontrol logic. This combination provides 360 measurements per part (ameasurement every degree of rotation) for unmatched repeatable profilemeasurements of part dimensional characteristics such as radii,concentricity, straightness, lengths, diameters, threads, tapers, etc. .. . .

Software: Part templates are created through an automatic databaseoperation for instant dimensions and tolerances of all pertinent partfeatures. With thousands of parts requiring inspection, this processsaves valuable set-up time.

Bottom Vision: Single view camera with advanced image analysis controllogic that minimizes surface and lighting variations. A strobe LED ringlight illuminates the bottom surface for dedicated inspection.

Inspections: Identify Head Markings and Inspect Recess/Drive Features.

MultiView Front Lighting detects surface imperfections such as—scratchgouges, flakes, burrs, smears, discoloration, chips, cracks, dents,damaged threads.

Using a sequenced lighting technique, milliseconds after completion ofthe 360 dimensional inspection, the parts are illuminated with top andbottom front LED lighting to provide the side cameras a view of the fullsurface of the part. As with the dimensional inspection, the VisionLabprocesses 360 surface inspection images (one image per every degree).

By processing 360 images, smaller surface defects are detected as theinspection window narrows to sections allowing more defined surfaceanalysis.

Software: Part templates are created through an automatic databaseoperation for instant visual defect detection. Defect analysis settingsare preprogrammed per Region of Interest (ROI), as shown in the threadedregion in the image of FIG. 22 b.

Top Vision: Single view camera with liquid lens technology and advancedimage analysis control logic that minimizes surface and lightingvariations. A strobe LED ring light illuminates the top surface fordedicated inspection.

By providing the liquid lens, neither the camera nor its lens assemblyis required to move for varying part lengths (i.e., the focal length ofthe liquid lens is electronically controlled).

Inspection: Top recess feature measurement and inspection.

Top Vision: Single view camera with liquid lens technology and advancedimage analysis control logic that minimizes surface and lightingvariations. A strobe LED ring light illuminates the internal surface forInner Diameter wall inspection including internal thread defectdetection.

By providing the liquid lens, neither the camera nor its lens assemblyis required to move for varying part lengths (i.e., the focal length ofthe liquid lens is electronically controlled).

Inspection: Inner diameter walls for visual defects and internal threadinspection.

Identifying Parts Process Via the Flowchart of FIG. 29

Using machine vision, software images are taken that will takemeasurements down to the micron level for each region of the part. Themeasurements are then compared with the data extracted version (i.e.templates) to find the closest matching parts. When a handful of partshave been found, attributes are used to narrow down to the best possiblechoice. Attributes include: Coating, Color, Identification, Head Type,Drive Type, Hole in Shank, Hole in Head and grip markings. A foreignpart check is done on the part to determine if all the measurements arewithin the tolerance levels. If any of the measurements are outside ofthe specified regions, it is determined that this part is not a partthat should be sorted, and the part will get dumped.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms of the invention. Rather,the words used in the specification are words of description rather thanlimitation, and it is understood that various changes may be madewithout departing from the spirit and scope of the invention.Additionally, the features of various implementing embodiments may becombined to form further embodiments of the invention.

What is claimed is:
 1. A method of inspecting unidentified mixed partsat an inspection station having a measurement axis to identify theparts, the method comprising: storing templates of a plurality of knowngood parts, each of the templates including a part profile includes arange of acceptable values and wherein each of the templates has a partidentification code associated therewith; placing a self-supporting partto be identified on a fixtureless rotary stage using a vision-basedrobotic system including a robot at the inspection station so that thepart axis is substantially centered and aligned with the measurementaxis; optically measuring a profile and features of the part as the partrotates at the inspection station; comparing the profile and thefeatures of the part to be identified with the profile and correspondingfeatures of the stored templates to identify a template which matchesthe profile and features of the part to be identified; and generatingand transmitting an identification signal representing the partidentification code for the part associated with the matched template.2. The method as claimed in claim 1, wherein the step of opticallymeasuring includes the steps of: projecting a beam of radiation at therotating part; rotating the rotary stage and the part to be identifiedabout the measurement axis so that the rotating part stays upright topartially obstruct the beam to obtain at least one unobstructed portionof the beam of radiation; and imaging the at least one unobstructedportion of the beam of radiation to obtain a first set of electricalsignals.
 3. The method as claimed in claim 2, further comprising:processing the first set of electrical signals to obtain the profile andfeatures of the part to be identified.
 4. The method as claimed in claim2, wherein the part to be identified is at least partially conductive orsemiconductive, wherein a feature of at least one of the templatesincludes an eddy current signature and wherein the method furthercomprises the steps of inducing an eddy current in the rotating part,sensing the induced eddy current and comparing the eddy currentsignature with the sensed eddy current.
 5. The method as claimed inclaim 2, wherein the part to be identified partially obstructs the beamof radiation to obtain first and second unobstructed portions andwherein the first and second unobstructed portions of the beam ofradiation are imaged to obtain the first set of electrical signals. 6.The method as claimed in claim 5, wherein each of the first and secondunobstructed portions of the beam of radiation contain a magnitude ofradiation which is representative of a respective geometric dimension ofthe part to be identified.
 7. The method as claimed in claim 1, whereinthe part to be identified has threads.
 8. The method as claimed in claim7, wherein at last one of the templates includes at least one featurerelated to threads.
 9. The method as claimed in claim 7, wherein thepart is an externally threaded fastener.
 10. The method as claimed inclaim 1, wherein each part axis is defined as being central to its partand parallel to its length.
 11. The method as claimed in claim 1,further comprising measuring the weight of the part on the rotary stage.12. The method as claimed in claim 1, wherein the part is made of ametal alloy and wherein the method further comprises determining thecomposition of the metal alloy.
 13. A system for inspecting unidentifiedmixed parts at an inspection station having a measurement axis toidentify the parts, each of the parts having a part axis, the systemcomprising: an electronic storage device to store templates of aplurality of known good parts, each of the templates including a partprofile and a set of features wherein each of the features includes arange of acceptable values and wherein each of the templates has a partidentification code associated therewith; a vision-based roboticsubsystem including a robot configured to pick up and place aself-supporting part to be identified on a fixtureless rotary stage atthe inspection station so that the part axis is substantially centeredand aligned with the measurement axis; a second subsystem to opticallymeasure a profile and features of the part during part rotation; and aprocessor operable to compare the profile and the features of the partto be identified with the profile and corresponding features of thestored templates to identify a template which matches the profile andfeatures of the part to be identified and to generate and transmit anidentification signal representing the part identification code for thepart associated with the matched template.
 14. The system as claimed inclaim 13, wherein the second subsystem comprises: a projector to projecta beam of radiation at the rotating part; a rotary actuator assembly torotate the rotary stage and the self-supporting part about themeasurement axis so that the rotating part stays upright to partiallyobstruct the beam to obtain at least one unobstructed portion of thebeam of radiation; and an imager to image the at least one unobstructedportion of the beam of radiation to obtain a first set of electricalsignals.
 15. The system as claimed in claim 14, wherein the processor isfurther operable to process the first set of electrical signals toobtain the profile and features of the part to be identified.
 16. Thesystem as claimed in claim 14, wherein the part to be identified is atleast partially conductive or semiconductive, wherein a feature of atleast one of the templates includes an eddy current signature andwherein the system further comprises at least one eddy current sensor toinduce an eddy current in the rotating part and to sense the inducededdy current, the processor also comparing the eddy current signaturewith the sensed eddy current.
 17. The system as claimed in claim 14,wherein the part to be identified partially obstructs the beam ofradiation to obtain first and second unobstructed portions and whereinthe first and second unobstructed portions of the beam of radiation areimaged to obtain the first set of electrical signals.
 18. The system asclaimed in claim 17, wherein each of the first and second unobstructedportions of the beam of radiation contain a magnitude of radiation whichis representative of a respective geometric dimension of the part to beidentified.
 19. The system as claimed in claim 13, further comprising aweight sensor coupled to the rotary state to measure the weight of apart on the rotary stage.
 20. The system as claimed in claim 13, whereinthe part includes a metal alloy and wherein the system further comprisesan x-ray fluorescence (XRF) metals analyzer to determine the compositionof the metal alloy.
 21. The system as claimed in claim 13, wherein thepart to be identified has threads.
 22. The system as claimed in claim21, wherein at least one of the templates includes at least one featurerelated to the threads.
 23. The system as claimed in claim 21, whereinthe part is an externally threaded fastener.
 24. The system as claimedin claim 13, wherein each part axis is defined as being central to itspart and parallel to its length.